This invention relates to an authoring system and an authoring method for supporting creation and editing of data in accordance with a preset scenario and, more particularly, to an authoring system and an authoring method for supporting creation and editing of a sequence of commands/data stating a preset operating pattern of a robot.
More particularly, it relates to an authoring system and an authoring method for supporting creation and editing of an operating pattern using a set of components prescribing the operating states of a robot, and to an authoring system and an authoring method for arraying the respective components on a computer display for supporting creation and editing of the operating pattern.
A mechanical apparatus for performing movements simulating the movement of the human being, using electrical or magnetic operation, is termed a xe2x80x9crobotxe2x80x9d. The etymology of the term robot is said to be xe2x80x9cROBOTAxe2x80x9d (slave machine) of the Slavic language. The robots started to be used widely in this country towards the end of the sixtieth. Most of the robots used were industrial robots, such as manipulators or transporting robots, aimed at automation or unmanned operations in plants.
In recent years, researches and development in legged mobile robots, including pet type robots, simulating the bodily mechanism or movements of animals, such as quadruples, e.g., dogs or cats, or so-called humanoid robots, simulating the bodily mechanism or movements of animals erected and walking on feet, such as human being, are progressing, and expectations are made of practical utilization of these types of robots. The legged mobile robots, while being unstable and difficult to control as to their orientation or walking, as compared to crawler or tire type robots, are superior in their performance in climbing up and down a ladder or a staircase, in riding over obstacles or walking or in running flexibly on a leveled or non-leveled terrain.
The standstill type robots, installed and used at a fixed place, such as armed robots, are in operation only in a stationary or local working space, such as for assembling or sorting of component parts. On the other hand, the mobile robots are not limited as to working space and are movable on a preset or undefined path in an unrestricted fashion to perform operations to take the place of human operators or to offer variegated services to take the place of the human being, dogs or other living organisms.
One of the usages of the legged mobile robot is substitution of a variety of operations dangerous or difficult to perform in industrial or productive activities. Examples of these usages include substitution of a variety of operations dangerous or difficult to perform, such as maintenance operations in nuclear power plants, thermal power plants or petrochemical plants, transporting or assembling operations in manufacturing plants, cleaning in high-rise buildings or rescuing e.g., on the site of conflagrations.
Another usage of the legged mobile robot is life-related usage, such as co-living with or rendering entertainment for the human being, rather than acting as the operational support described above. This sort of robot emulates a rich feeling expression exploiting the four limbs or an operation mechanism of the legged walking animal of high intellect, such as human being or dog (pet). Moreover, this sort of the robot is requested not only to execute the pre-set operating pattern measly faithfully, but also to realize vivid expressions dynamically responding to the language or behavior of a counterpart operator, such as xe2x80x98praisingxe2x80x99, xe2x80x98scoldingxe2x80x99 or xe2x80x98pattingxe2x80x99.
With a conventional toy machine, the relationship between the user operation and the responsive operation is fixed, such that the toy""s movements cannot be changed to suit to the user""s liking. The result is that the user gets tired of the toy, simply repeating only the same sort of the operations, sooner or later.
Conversely, an intellectual robot owns a behavioral or learning model, emanating from movements, and determines its movements as it varies the model based on the input information from outside, such as speech, image or tactile feeling, to realize autonomous thinking or operation control. The feeling or the autonomous behavior of the robot itself can be represented by providing the robot with the feeling or instinct model. By the robot owning a picture inputting device or with a speech input/output device and executing the processing of picture or speech recognition, it is possible to realize realistic communication between the robot and the human being on a higher intellectual level.
In addition, by affording the possibility of changing the model responsive to the detection of a stimulus from outside, such as user""s actuation, that is the xe2x80x98learning effectxe2x80x99, it is possible to present to the user a movement pattern which is not tiresome to the user or which adapts itself more satisfactorily to the user""s liking.
Nowadays, the legged mobile robot owns a high information processing capability, such that the robot itself may be thought of as a sort of a computing system. Consequently, the sequence of the highly advanced sophisticated operations, constructed by the operating pattern, realized on the robot, or by the combination of plural basic operating patterns, may be constructed by an operation similar to that performed in the computer programming.
In near future, it may be anticipated that the robot becomes more popular to become more diffused not only in industrial circles but also in households and in our everyday life. In particular, as concerns products which pursue entertainment aspects, it may be anticipated that consumers at large, not having profound knowledge in computer or computer programming, purchase and use the robot. It may be thought to be desirable to provide the users at large with a tool which helps create and edit the operating sequence of the robot relatively readily and efficiently by interactive processing, that is with a so-called authoring system.
It is therefore an object of the present invention to provide an excellent authoring system and method which are able to help create and edit a sequence of command/data stating a preset operating pattern of the robot.
It is another object of the present invention to provide an excellent authoring system and method which are able to help create and edit an operating pattern using a set of components prescribing the operating states of a robot.
It is a further object of the present invention to provide an excellent authoring system and method which allow to array various components on a computer display to help create and edit an operating pattern for the robot.
An authoring system according to the present invention is such a system supporting the creation and editing of an action of a multi-joint structure comprised of a combination of a plurality of chronological data, including a user inputting unit for inputting commands or data from a user, a user presenting unit for presenting an editing area in which chronological data making up the action are arrayed chronologically along the time axis, and a chronological data editing unit provided for each chronological data making up the action. The chronological data editing unit creates or edits relevant chronological data based on a user input through the user presenting unit.
The xe2x80x98systemxe2x80x99 herein means a logical assembly of plural devices or functional modules realizing specified functions, without regard to whether or not the individual devices or functional modules are enclosed in one casing.
The multi-joint structure is a legged robots having two legs, four legs and so forth and which are constructed from plural joint actuators. Alternatively, the multi-joint structure may be other types of multi-joint robots, or a character for producing an animation, based on joint movements, as a result of computer graphics.
The chronological data making up the action may be motion data stating the chronological movements of respective joints of the multi-joint structure. The motion data may prescribe chronological movements of respective joints of the multi-joint structure and, by arraying two or more keyframes, each representing the multi-joint structure striking a preset pose on the time axis, may smoothly join the respective keyframes.
Another chronological data making up the action is sound data output as sound in timed relation to reproduction of the action. The sound data may be stated in the MIDI (Musical Instrumental Digital Interface) form or WAVE form.
Still another example of the chronological data making up the action is indicator display data stating the turning on/off of display indicators displayed in timed relation to action reproduction. The indicator display data may be stated in the MIDI (Musical Instrumental Digital Interface) form.
The editing area presented by the user presenting unit may be comprised of chronological-data-based chronological data display channels. These data extend along the horizontally arrayed time axis, with the display channels being arrayed in the vertical direction.
That is, with the authoring system according to the present invention, the chronological data making up an action of the mobile robot may be displayed as they are arrayed along the time axis on the time table of the two-dimensional time line style. Consequently, the creation and editing may be prosecuted as synchronization between the respective chronological data is visually checked so that a working environment efficient and intuitively comprehensible may be provided for action editing.
The editing area presented by the user presenting unit may include one or more time display lines extending in the vertical direction for specifying the relevant time as prescribed by the time ruler. In such case, the state of progress between the respective time indicating lines or the state of synchronization between the respective chronological data channels may be visually checked with the time indicating lines as a reference.
The editing area presented by the user presenting unit may include a vertically extending current time display line for indicating the current time on the time axis prescribed by the time ruler. The current time display line may be configured for being shifted on the time ruler to a position corresponding to user actuation.
The editing area presented by the user presenting unit may include a display window for previewing movements of the multi-joint structure at the current time.
The editing area presented by the user presenting unit may include a keyframe channel for indicating the respective keyframes or thumbnails thereof along the time axis as prescribed by the time ruler. In the keyframe channel, the time changes of the keyframe may be accommodated so as to follow up with keyframe or thumbnail dragging. A pose editing picture for editing a relevant pose may be started responsive to the operation of selection by the user of the keyframe or thumbnail within the keyframe channel.
The editing area presented by the user presenting unit may also include a motion channel for editing and displaying the contents of a motion along the time axis prescribed by the time ruler. This motion channel is constructed by arraying respective timing charts, representing the chronological movements of respective joints of the multi-joint structure in the vertical direction. It is also possible to accommodate changes in the joint movements at a relevant time point so as to follow up with dragging on the timing chart in the motion channel.
The editing area presented by the user presenting unit may also be provided with a sound channel for indicating the contents of the sound along the time axis as prescribed by the time ruler, or with a display indicator channel for displaying the contents of the indicator display data along the time axis prescribed by the time ruler.
The user presenting unit may also be provided with a sound editing area for displaying and editing the sound contents along the time axis prescribed by the time ruler.
The sound editing area may include a score channel constituted by a basic grid along the time axis direction and by a piano keyboard. In this case, a score is formed by the sound duration as the reference of the time axis and the pitch of the piano keys. On the score channel, the sound may be edited by arraying the color corresponding to sound notes in a cell corresponding to the desired time point and the sound scale. The velocity channel may also be demonstrated for representing the intensity of each sound along the time axis.
The user presenting unit may further include an indicator editing area for displaying and editing the contents of indicator data along the time axis prescribed by the time ruler.
The indicator editing area includes a score channel constituted by a list of sites of arraying of display indicators and by the basic grid along the time axis direction. In such case, a score may be edited from one site list to another by indicating the turned-on state of the indicator in each site on the time axis on a score channel.
The user presenting unit may further display a preview window for visually checking an action of the multi-joint structure generated based on each chronological data as edited by the chronological data editing unit.
The preview window may include a 3D view area for three-dimensional display of movements of the multi-joint structure generated based on motion data. The preview window may include a display indicator operation preview area for displaying the operation of the indicators corresponding to the indicator display data in synchronism with the preview of other chronological data.
The user presenting unit may further display a pose window for editing the pose of the multi-joint structure by the GUI operation.
The pose window may include an entity specifying area for displaying the multi-joint structure in a developed plan view for accepting user selection of editable sites. The authoring pose window may include a list specifying area for displaying a list of editable sites of the multi-joint structure and setting values thereof. The pose window may also include a setting value area for displaying a list of the names of setting sites, setting values, settable maximum values and settable minimum values of editable sites of the multi-joint structure. The pose window may also include a 3D display area for 3D displaying a full image of the multi-joint structure generated by 3D graphics and for accepting user selection on the 3D display.
The authoring system may also include data inputting means for inputting chronological data making up an action of the multi-joint structure from outside. The pose window may also display the pose generated based on data input from the data inputting means.
The user presenting unit may further display a motion preview window arraying one or more key frames making up the motion or thumbnails thereof in accordance with the chronological sequence used in motion reproduction.
An authoring method according to the present invention is such a method for supporting the creation and editing of an action of a multi-joint structure comprised of a combination of a plurality of chronological data, and includes a user presenting step of presenting an editing area in which the chronological data making up the action are arrayed chronologically along the time axis and a chronological data editing step for creating or editing relevant chronological data based on a user input through the editing area, provided for each chronological data making up the action, via the user presenting step.
One of the chronological data making up the action is motion data stating the chronological movements of respective joints of the multi joint structure. This motion data may prescribe chronological movements of respective joints of the multi-joint structure. The motion data smoothly joins two or more keyframes by arraying these keyframes each representing the multi-joint structure striking a preset pose on the time axis.
Another example of the chronological data making up the action is sound data output as sound in timed relation to reproduction of the action. The sound data may be stated in the MIDI (Musical Instrumental Digital Interface) form or WAVE form. Still another example of chronological data making up the action is display indicator display data stating the turning on/off of display indicators, with the display data being displayed in timed relation to reproduction of the action. The indicator display data may be stated in the MIDI (Musical Instrumental Digital Interface) form.
The editing area presented in the user presenting step may be comprised of chronological-data-based chronological data display channels, with the data extending along the horizontally arrayed time axis, the channels being arrayed in the vertical direction.
That is, with the authoring method according to the present invention, the chronological data forming the action of the mobile robot, such as motion data, sound data or LED operating data, may be arrayed and displayed along the time axis on the timetable of the two-dimensional time line style to render it possible to visually check the synchronization between respective chronological data in prosecuting the creation or editing. The result is provision of a working environment for action edition which is more efficient and comprehensible intuitively.
The editing area presented in the user presenting step may include a time ruler comprised of graduations used to display the time axis in real-time. In such case, it is possible to readily visually recognize the synchronization between the respective chronological data channels and the time axis.
The editing area presented by the user presenting step may include one or more time display lines extending in the vertical direction for indicating the relevant time prescribed by a time ruler. In such case, the state of progress between the respective time indicating lines or the state of synchronization between the respective chronological data channels may be visually checked with the time indicating lines as a reference.
The user presenting step may present an editing area having a vertically extending current time display line for indicating the current time on the time axis prescribed by the time ruler. There may further be included a further step of shifting the current time display line on the time ruler to a position corresponding to user actuation.
There may further be provided a step of presenting a display window for previewing movements of the multi-joint structure at the current time.
The user presenting step may further present an editing area having a keyframe channel for demonstrating each keyframe or its thumbnail in accordance with the time axis prescribed by the time ruler. In such case, there may further be provided a step of changing the keyframe time so as to follow up with dragging of the keyframe or its thumbnail in the keyframe channel. There may also be provided a step of starting a pose editing picture for editing the relevant pose responsive to the user""s operation for selecting the keyframe or its thumbnail in the keyframe channel.
The user presenting step may also present an editing area provided with a motion channel for editing and displaying the motion contents along the time axis prescribed by the time ruler. In such case, the respective timing charts representing the chronological movements of the respective joints of the multi-joint structure may be arrayed vertically to represent the motion channel. There may further be provided a step of changing the joint movements at the relevant time points such as to follow up with dragging on the timing charts in the motion channel.
In the user presenting step, there may be presented an editing area having a sound channel for demonstrating the sound contents along the time axis prescribed by the time ruler. Alternatively, the user presenting step may present an editing area having a display indicator channel for displaying the contents of the indicator display data along the time axis prescribed by the time ruler.
The authoring method may further include a step of displaying the sound editing area for displaying and editing the sound contents along the time axis prescribed by the time ruler.
The sound editing area may include a score channel constituted by a basic grid along the time axis direction and a piano keyboard. In such case, a score is formed by the sound duration as the reference of the time axis and by the pitch of the piano keys. On the score channel, the sound may be edited by arraying the color corresponding to sound notes in a cell corresponding to the desired time point and the sound scale on the score channel. The sound editing area may also include a velocity channel for representing the intensity of each sound along the time axis.
The authoring method may further include a step of demonstrating a display indicator editing area for displaying and editing the contents of the indicator display area along the time axis prescribed by the tie ruler.
The indicator editing area may include a score channel constituted by a list of arraying sites of the display indicators and by the basic grid along the time axis direction. In such case, the score may be edited from one site list to another by demonstrating the turned-on state of the display indicators in respective sites on the time axis on a score channel.
The authoring method may further include a step of displaying a preview window for visually checking an action of the multi-joint structure generated based on each chronological data as edited in the chronological data editing step.
The preview window indicating step may three-dimensionally display movements of the multi-joint structure, generated based on motion data, in a 3D view area. Alternatively, the preview window indicating step may display the operation of the display indicators corresponding to the indicator display data in synchronism with the preview of other chronological data.
The authoring method may further include a step of displaying a pose window for editing the pose of the multi-joint structure by the GUI operation.
The pose window may include an entity specifying area for displaying the multi-joint structure in a developed plan view for accepting user selection of editable sites, a list specifying area for displaying a list of editable sites of the multi-joint structure and setting values thereof, a setting value area for displaying a list of the names of setting sites, setting values, settable maximum values and settable minimum values of editable sites of the multi-joint structure, or a 3D display area for 3D demonstrating a full image of the multi-joint structure generated by 3D graphics and for accepting user selection of the editable sites on the 3D display.
The authoring method may further include a data inputting step of inputting chronological data making up an action of the multi-joint structure from outside. The pose window displays the pose generated based on data input from the data inputting step.
The authoring method may also include a step of displaying a motion preview window arraying one or more key frames making up the motion or thumbnails thereof in accordance with the chronological sequence used in motion reproduction.
A recording medium according to the present invention has physically stored thereon, in a computer-readable form, a computer software stating the processing for supporting the creation and editing of an action of a multi-joint structure, comprised of the combination of plural chronological data, on a computer system. The computer software includes a user presenting step of presenting an editing area in which chronological data making up the action are arrayed chronologically along the time axis, and a chronological data editing step of creating or editing relevant chronological data based on a user input through an editing area by the user presenting step. The editing area is provided for each chronological data making up the action.
The recording medium according to the present invention is a medium for physically furnishing a computer software in a computer readable form to the general-purpose computer system capable of executing a variety of program codes. Such medium is a removable portable recording medium, exemplified by a CD (Compact Disc), an FD (Floppy Disc), a MD (Mini-Disc) or a MO (Magneto-Optical Disc). It is also possible to furnish the computer software to a specified computer system in a computer readable form through a transmission medium, such as a network, which may be a wireless or cable network.
This recording medium represents a definition of the cooperative relationship between the computer software an the recording medium in structure or function in order to achieve the functions of the preset computer software on the computer system. Stated differently, by installing a preset compute software on a computer system through a recording medium according to the third aspect of the present invention, the cooperative action may be manifested on the computer system to achieve the operation and effect similar to those of the authoring system and method in the first and second aspects of the present invention.
Other objects, features and advantages of the present invention will become more apparent from reading the embodiments of the present invention as shown in the drawings.